Improving Semi-Global Matching: Cost aggregation and confidence measure

Digital elevation models are one of the basic products that can be generated from remotely sensed imagery. The Semi Global Matching (SGM) algorithm is a robust and practical algorithm for dense image matching. The connection between SGM and Belief Propagation was recently developed, and based on that improvements such as correction of over-counting the data term, and a new confidence measure have been proposed. Later the MGM algorithm has been proposed, it aims at improving the regularization step of SGM, but has only been evaluated on the Middlebury stereo benchmark so far. This paper evaluates these proposed improvements on the ISPRS satellite stereo benchmark, using a Pleiades Triplet and a Cartosat-1 Stereo pair. The over-counting correction slightly improves matching density, at the expense of adding a few outliers. The MGM cost aggregation shows leads to a slight increase of accuracy

A Framework for SAR-Optical Stereogrammetry over Urban Areas

Currently, numerous remote sensing satellites provide a huge volume of diverse earth observation data. As these data show different features regarding resolution, accuracy, coverage, and spectral imaging ability, fusion techniques are required to integrate the different properties of each sensor and produce useful information. For example, synthetic aperture radar (SAR) data can be fused with optical imagery to produce 3D information using stereogrammetric methods. The main focus of this study is to investigate the possibility of applying a stereogrammetry pipeline to very-high-resolution (VHR) SAR-optical image pairs. For this purpose, the applicability of semi-global matching is investigated in this unconventional multi-sensor setting. To support the image matching by reducing the search space and accelerating the identification of correct, reliable matches, the possibility of establishing an epipolarity constraint for VHR SAR-optical image pairs is investigated as well. In addition, it is shown that the absolute geolocation accuracy of VHR optical imagery with respect to VHR SAR imagery such as provided by TerraSAR-X can be improved by a multi-sensor block adjustment formulation based on rational polynomial coefficients. Finally, the feasibility of generating point clouds with a median accuracy of about 2m is demonstrated and confirms the potential of 3D reconstruction from SAR-optical image pairs over urban areas.Comment: This is the pre-acceptance version, to read the final version, please go to ISPRS Journal of Photogrammetry and Remote Sensing on ScienceDirec

Quantitative phase determination by using a Michelson interferometer

The Michelson interferometer is one of the best established tools for quantitative interferometric measurements. It has been, and is still successfully used, not only for scientific purposes, but it is also introduced in undergraduate courses for qualitative demonstrations as well as for quantitative determination of several properties such as refractive index, wavelength, optical thickness, etc. Generally speaking, most of the measurements are carried out by determining phase distortions through the changes in the location and/or shape of the interference fringes. However, the extreme sensitivity of this tool, for which minimum deviations of the conditions of its branches can cause very large modifications in the fringe pattern, makes phase changes difficult to follow and measure. The purpose of this communication is to show that, under certain conditions, the sensitivity of the Michelson interferometer can be 'turned down' allowing the quantitative measurement of phase changes with relative ease. As an example we present how the angle (or, optionally, the refractive index) of a transparent standard optical wedge can be determined. Experimental results are shown and compared with the data provided by the manufacturer showing very good agreement.Fil: Pomarico, Juan Antonio. Consejo Nacional de Investigaciones Científicas y Técnicas. Centro Científico Tecnológico Conicet - Tandil; Argentina. Universidad Nacional del Centro de la Provincia de Buenos Aires; ArgentinaFil: Molina, Pablo Fernando. Universidad Nacional del Centro de la Provincia de Buenos Aires; Argentina. Consejo Nacional de Investigaciones Científicas y Técnicas. Centro Científico Tecnológico Conicet - Tandil; ArgentinaFil: D'angelo, Cristian Adrián. Universidad Nacional del Centro de la Provincia de Buenos Aires; Argentina. Consejo Nacional de Investigaciones Científicas y Técnicas. Centro Científico Tecnológico Conicet - Tandil; Argentin

Geometric Evaluation of Gaofen-7 Stereo Data

China's first sub-metre stereo satellite, GaoFen-7, was launched on 7 November 2019. One of the main criteria for a stereo mapping satellite is the geometric accuracy of the images. In this paper, we present a systematic evaluation of the geometry accuracy of Gaofen-7 on two scenes over the centre of Munich, Germany. The geometry accuracy is evaluated in a three-step workflow: 1) direct georeferencing accuracy; 2) image orientation using bundle adjustment with ground control points; 3) height accuracy of the generated digital surface model (DSM). In addition to dense LiDAR point clouds, ground control points were measured in the field. These were used as references. The results show that RPC bundle adjustment with 0 order bias correction is sufficient to achieve sub-metre absolute accuracy. The height accuracy of the generated digital surface models varies with land cover type, ranging from 0.9m (NMAD) in open areas to 4.5m in urban areas

Multi-label learning based semi-global matching forest

Semi-Global Matching (SGM) approximates a 2D Markov Random Field (MRF) via multiple 1D scanline optimizations, which serves as a good trade-off between accuracy and efficiency in dense matching. Nevertheless, the performance is limited due to the simple summation of the aggregated costs from all 1D scanline optimizations for the final disparity estimation. SGM-Forest improves the performance of SGM by training a random forest to predict the best scanline according to each scanline’s disparity proposal. The disparity estimated by the best scanline acts as reference to adaptively adopt close proposals for further post-processing. However, in many cases more than one scanline is capable of providing a good prediction. Training the random forest with only one scanline labeled may limit or even confuse the learning procedure when other scanlines can offer similar contributions. In this paper, we propose a multi-label classification strategy to further improve SGM-Forest. Each training sample is allowed to be described by multiple labels (or zero label) if more than one (or none) scanline gives a proper prediction. We test the proposed method on stereo matching datasets, from Middlebury, ETH3D, EuroSDR image matching benchmark, and the 2019 IEEE GRSS data fusion contest. The result indicates that under the framework of SGM-Forest, the multi-label strategy outperforms the single-label scheme consistently

The TUM-DLR Multimodal Earth Observation Evaluation Benchmark

We present a new dataset for development, benchmarking, and evaluation of remote sensing and earth observation approaches with special focus on converging perspectives. In order to provide data with different modalities, we observed the same scene using satellites, airplanes, unmanned aerial vehicles (UAV), and smartphones. The dataset is further complemented by ground-truth information and baseline results for different application scenarios. The provided data can be freely used by anybody interested in remote sensing and earth observation and will be continuously augmented and updated

Benchmarking and quality analysis of DEM generated from high and very high resolution optical stereo satellite data

The Working Group 4 of Commission I on ¿Geometric and Radiometric Modelling of Optical Spaceborne Sensors¿ provides on its website several stereo data sets from high and very high resolution spaceborne sensors. Among these are data from the 2.5 meter class like ALOS-PRISM and Cartosat-1 as well as, in near future, data from the highest resolution sensors (0.5 m class) like GeoEye-1 and Worldview-1 and -2. The region selected is an area in Catalonia, Spain, including city areas (Barcelona), rural areas and forests in flat and medium undulated terrain as well as steeper mountains. In addition to these data sets, ground truth data: orthoimages from airborne campaigns and Digital Elevation Models (DEM) produced by laser scanning, all data generated by the Institut Cartogràfic de Catalunya (ICC), are provided as reference for comparison. The goal is to give interested scientists of the ISPRS community the opportunity to test their algorithms on DEM generation, to see how they match with the reference data and to compare their results within the scientific community. A second goal is to develop further methodology for a common DEM quality analysis with qualitative and quantitative measures. Several proposals exist already and the working group is going to publish them on their website. But still there is a need for more standardized methodologies to quantify the quality even in cases where no better reference is available. The data sets, the goal of the benchmarking and first evaluation results are presented within the paper. Algorithms using area-based least squares matching are compared to those using additionally feature-based matching or newly developed algorithms from the Computer Vision community. The main goal though is to motivate further researchers to join the benchmarking and to discuss pros and cons of the methods as well as to trigger the process of establishing standardized DEM quality figures and procedures.JRC.DG.G.2-Global security and crisis managemen

SyntCities: A Large Synthetic Remote Sensing Dataset for Disparity Estimation

Studies in the last years have proved the outstanding performance of deep learning for computer vision tasks in the remote sensing field, such as disparity estimation. However, available datasets mostly focus on close-range applications like autonomous driving or robot manipulation. To reduce the domain gap while training we present SyntCities, a synthetic dataset resembling the aerial imagery on urban areas. The pipeline used to render the images is based on 3-D modeling, which helps to avoid acquisition costs, provides subpixel accurate dense ground truth and simulates different illumination conditions. The dataset additionally provides multiclass semantic maps and can be converted to point cloud format to benefit a wider research community. We focus on the task of disparity estimation and evaluate the performance of the traditional semiglobal matching and state-of-the-art architectures, trained with SyntCities and other datasets, on real aerial and satellite images. A comparison with the widely used SceneFlow dataset is also presented. Strategies using a mixture of both real and synthetic samples are studied as well. Results show significant improvements in terms of accuracy for the disparity maps

An Evaluation of Stereo and Multiview Algorithms for 3d Reconstruction with Synthetic Data

The reconstruction of 3D scenes from images has usually been addressed with two different strategies, namely stereo and multi-view. The former requires rectified images and generates a disparity map, while the latter relies on the camera parameters and directly retrieves a depth map. For both cases, deep learning architectures have shown an outstanding performance. However, due to the differences between input and output data, the two strategies are difficult to compare on a common scene. Moreover, for remote sensing applications multi-view data is hard to acquire and the ground truth is either sparse or affected by outliers. Hence, in this article we evaluate the performance of stereo and multi-view architectures trained on synthetic data resembling remote sensing images. The data has been and processed and organized to be compatible with both kind of neural networks. For a fair comparison, training and testing are done only with two views. We focus on the accuracy of the reconstruction, as well as the impact of the depth range and the baseline of the stereo array. Results are presented for deep learning architectures and non-learning algorithms

DSM2DTM: An End-to-End Deep Learning Aproach for Digital Terrain Model Generation

Remotely sensed Earth elevation data or digital surface model (DSM) typically contains both terrain and above-ground information such as vegetation and man-made constructions. However, many applications require pure bare-terrain data, also known as digital terrain model (DTM). But how do we separate 3D objects on the DSM from the ground? The most commonly used approaches are still based on various filtering techniques, which in turn involve the pre-definition of thresholds or specific parameters depending on the inhomogeneity of the scene. Despite many long existing and newly developed approaches the general fully automatic extraction of large-scale, reliable DTMs is still a problem – especially the preservation of steep terrain features in terraced landscapes. In this context, we explore several deep learning models and select one based on the EfficientNet architecture. This model serves as an encoder in the UNet-shaped framework and – despite its relatively low amount of parameters compared to common network architectures – it can automatically distinguish non-ground pixels and estimate the bare-ground height information while maintaining the complexity of the anthropogenic geomorphology of landscapes. In a series of experiments, we demonstrate that the DTM generated with the proposed method significantly outperforms other DTM generation approaches – both quantitatively and qualitatively